Face Generation¶

In this project, you'll use generative adversarial networks to generate new images of faces.

Get the Data¶

You'll be using two datasets in this project:

MNIST
CelebA

Since the celebA dataset is complex and you're doing GANs in a project for the first time, we want you to test your neural network on MNIST before CelebA. Running the GANs on MNIST will allow you to see how well your model trains sooner.

If you're using FloydHub, set data_dir to "/input" and use the FloydHub data ID "R5KrjnANiKVhLWAkpXhNBe".

#data_dir = './data'

# FloydHub - Use with data ID "R5KrjnANiKVhLWAkpXhNBe"
data_dir = '/input'


"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
import helper

helper.download_extract('mnist', data_dir)
helper.download_extract('celeba', data_dir)

Found mnist Data
Found celeba Data

Explore the Data¶

MNIST¶

As you're aware, the MNIST dataset contains images of handwritten digits. You can view the first number of examples by changing show_n_images.

show_n_images = 25

"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
%matplotlib inline
import os
from glob import glob
from matplotlib import pyplot

mnist_images = helper.get_batch(glob(os.path.join(data_dir, 'mnist/*.jpg'))[:show_n_images], 28, 28, 'L')
pyplot.imshow(helper.images_square_grid(mnist_images, 'L'), cmap='gray')

<matplotlib.image.AxesImage at 0x7f63af1a0630>

CelebA¶

The CelebFaces Attributes Dataset (CelebA) dataset contains over 200,000 celebrity images with annotations. Since you're going to be generating faces, you won't need the annotations. You can view the first number of examples by changing show_n_images.

show_n_images = 25

"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
mnist_images = helper.get_batch(glob(os.path.join(data_dir, 'img_align_celeba/*.jpg'))[:show_n_images], 28, 28, 'RGB')
pyplot.imshow(helper.images_square_grid(mnist_images, 'RGB'))

<matplotlib.image.AxesImage at 0x7f63af0e0710>

Preprocess the Data¶

Since the project's main focus is on building the GANs, we'll preprocess the data for you. The values of the MNIST and CelebA dataset will be in the range of -0.5 to 0.5 of 28x28 dimensional images. The CelebA images will be cropped to remove parts of the image that don't include a face, then resized down to 28x28.

The MNIST images are black and white images with a single color channel while the CelebA images have 3 color channels (RGB color channel).

Build the Neural Network¶

You'll build the components necessary to build a GANs by implementing the following functions below:

model_inputs
discriminator
generator
model_loss
model_opt
train

Check the Version of TensorFlow and Access to GPU¶

This will check to make sure you have the correct version of TensorFlow and access to a GPU

"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
from distutils.version import LooseVersion
import warnings
import tensorflow as tf

# Check TensorFlow Version
assert LooseVersion(tf.__version__) >= LooseVersion('1.0'), 'Please use TensorFlow version 1.0 or newer.  You are using {}'.format(tf.__version__)
print('TensorFlow Version: {}'.format(tf.__version__))

# Check for a GPU
if not tf.test.gpu_device_name():
    warnings.warn('No GPU found. Please use a GPU to train your neural network.')
else:
    print('Default GPU Device: {}'.format(tf.test.gpu_device_name()))

TensorFlow Version: 1.1.0
Default GPU Device: /gpu:0

Input¶

Implement the model_inputs function to create TF Placeholders for the Neural Network. It should create the following placeholders:

Real input images placeholder with rank 4 using image_width, image_height, and image_channels.
Z input placeholder with rank 2 using z_dim.
Learning rate placeholder with rank 0.

Return the placeholders in the following the tuple (tensor of real input images, tensor of z data)

import problem_unittests as tests

def model_inputs(image_width, image_height, image_channels, z_dim):
    """
    Create the model inputs
    :param image_width: The input image width
    :param image_height: The input image height
    :param image_channels: The number of image channels
    :param z_dim: The dimension of Z
    :return: Tuple of (tensor of real input images, tensor of z data, learning rate)
    """
    # TODO: Implement Function
    input_real = tf.placeholder(tf.float32, [None, image_width, image_height, image_channels], name='input_real')
    input_z = tf.placeholder(tf.float32, [None, z_dim], name='input_z')
    learning_rate = tf.placeholder(tf.float32, None, name='learning_rate')
    return input_real, input_z, learning_rate


"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_model_inputs(model_inputs)

Tests Passed

Discriminator¶

Implement discriminator to create a discriminator neural network that discriminates on images. This function should be able to reuse the variables in the neural network. Use tf.variable_scope with a scope name of "discriminator" to allow the variables to be reused. The function should return a tuple of (tensor output of the discriminator, tensor logits of the discriminator).

def leaky_relu(x, alpha=0.2, name='leaky_relu'):
    return tf.maximum(alpha * x, x, name=name)

def conv_layer2d(inputs, filters, kernel_size=5, strides=2, padding='same', normalize=True):
    conv_layer2d = tf.layers.conv2d(inputs=inputs, filters=filters, kernel_size=kernel_size,
                                  strides=strides, padding=padding, 
                                  kernel_initializer=tf.contrib.layers.xavier_initializer())
    if normalize:
        conv_layer2d = tf.layers.batch_normalization(inputs=conv_layer2d, training=True)
    conv_layer2d = leaky_relu(x=conv_layer2d)
    conv_layer2d = tf.layers.dropout(inputs=conv_layer2d, rate=0.50)
    return conv_layer2d

def get_conv_layer_volume(conv_layer):
    return conv_layer.get_shape().as_list()[1] * conv_layer.get_shape().as_list()[2] * conv_layer.get_shape().as_list()[3]

def reshape_conv_layer(conv_layer, conv_layer_volume):
    return tf.reshape(conv_layer, [-1, conv_layer_volume])

def discriminator(images, reuse=False, alpha=0.2):
    """
    Create the discriminator network
    :param images: Tensor of input image(s)
    :param reuse: Boolean if the weights should be reused
    :return: Tuple of (tensor output of the discriminator, tensor logits of the discriminator)
    """
    # TODO: Implement Function
    with tf.variable_scope('discriminator', reuse=reuse):
        x = conv_layer2d(inputs=images, filters=32, normalize=False)
        conv_layer = conv_layer2d(inputs=x, filters=64)
        conv_layer = conv_layer2d(inputs=conv_layer, filters=128)
        #conv_layer = conv_layer2d(inputs=conv_layer, filters=128)
        conv_layer_volume = get_conv_layer_volume(conv_layer=conv_layer)
        conv_layer_flattened = reshape_conv_layer(conv_layer=conv_layer, conv_layer_volume=conv_layer_volume)
        logits = tf.layers.dense(inputs=conv_layer_flattened, units=1)
        out = tf.sigmoid(x=logits)
        
        return out, logits

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_discriminator(discriminator, tf)

Tests Passed

Generator¶

Implement generator to generate an image using z. This function should be able to reuse the variables in the neural network. Use tf.variable_scope with a scope name of "generator" to allow the variables to be reused. The function should return the generated 28 x 28 x out_channel_dim images.

def generator_first_layer(inputs, filters, is_train, width=7, height=7):
    x = tf.layers.dense(inputs=inputs, units=width*height*filters)
    x = tf.reshape(x, shape=[-1, width, height, filters])
    x = tf.layers.batch_normalization(inputs=x, training=is_train)
    x = leaky_relu(x=x)
    return x

def conv2d_transpose(inputs, filters, is_train, kernel_size=5, strides=2, padding='same', normalize=True):
    conv_layer_transposed = tf.layers.conv2d_transpose(inputs=inputs, filters=filters, kernel_size=kernel_size,
                                                      strides=strides, padding=padding,
                                                      kernel_initializer=tf.contrib.layers.xavier_initializer())
    if normalize:
        conv_layer_transposed = tf.layers.batch_normalization(inputs=conv_layer_transposed, training=is_train)
    conv_layer_transposed = leaky_relu(x=conv_layer_transposed)
    #conv_layer_transpose = tf.layers.dropout(inputs=conv_layer_transpose, rate=0.50, training=is_train)

    return conv_layer_transposed

def generator(z, out_channel_dim, is_train=True, alpha=0.2):
    """
    Create the generator network
    :param z: Input z
    :param out_channel_dim: The number of channels in the output image
    :param is_train: Boolean if generator is being used for training
    :return: The tensor output of the generator
    """
    # TODO: Implement Function
    with tf.variable_scope('generator', reuse=not is_train):
        x = generator_first_layer(inputs=z, filters=256, is_train=is_train)
        conv_layer_transpose = conv2d_transpose(inputs=x, filters=128, is_train=is_train)
        conv_layer_transpose = conv2d_transpose(inputs=conv_layer_transpose, filters=64, is_train=is_train)
        #conv_layer_transpose = conv2d_transpose(inputs=conv_layer_transpose, filters=32, is_train=is_train, 
        #                                         strides=(1, 1))

        logits = conv2d_transpose(inputs=conv_layer_transpose, filters=out_channel_dim, is_train=is_train,
                                  strides=(1, 1), normalize=False)
        out = tf.tanh(x=logits)
        
        return out

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_generator(generator, tf)

Tests Passed

Loss¶

Implement model_loss to build the GANs for training and calculate the loss. The function should return a tuple of (discriminator loss, generator loss). Use the following functions you implemented:

discriminator(images, reuse=False)
generator(z, out_channel_dim, is_train=True)

def model_loss(input_real, input_z, out_channel_dim):
    """
    Get the loss for the discriminator and generator
    :param input_real: Images from the real dataset
    :param input_z: Z input
    :param out_channel_dim: The number of channels in the output image
    :return: A tuple of (discriminator loss, generator loss)
    """
    # TODO: Implement Function
    smooth = 0.1
    g_model = generator(z=input_z, out_channel_dim=out_channel_dim)
    d_model_real, d_logits_real = discriminator(images=input_real)
    d_model_fake, d_logits_fake = discriminator(images=g_model, reuse=True)
    
    g_loss = tf.reduce_mean(
        tf.nn.sigmoid_cross_entropy_with_logits(logits=d_logits_fake, 
                                             labels=tf.ones_like(d_model_fake)))
    d_loss_real = tf.reduce_mean(
        tf.nn.sigmoid_cross_entropy_with_logits(logits=d_logits_real,
                                             labels=tf.ones_like(d_model_real) * (1.0 - smooth)))
    d_loss_fake = tf.reduce_mean(
        tf.nn.sigmoid_cross_entropy_with_logits(logits=d_logits_fake,
                                             labels=tf.zeros_like(d_model_fake)))
    
    d_loss = d_loss_real + d_loss_fake
    
    return d_loss, g_loss


"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_model_loss(model_loss)

Tests Passed

Optimization¶

Implement model_opt to create the optimization operations for the GANs. Use tf.trainable_variables to get all the trainable variables. Filter the variables with names that are in the discriminator and generator scope names. The function should return a tuple of (discriminator training operation, generator training operation).

def model_opt(d_loss, g_loss, learning_rate, beta1):
    """
    Get optimization operations
    :param d_loss: Discriminator loss Tensor
    :param g_loss: Generator loss Tensor
    :param learning_rate: Learning Rate Placeholder
    :param beta1: The exponential decay rate for the 1st moment in the optimizer
    :return: A tuple of (discriminator training operation, generator training operation)
    """
    # TODO: Implement Function
    trainable_vars = tf.trainable_variables()
    d_vars = [var for var in trainable_vars if var.name.startswith('discriminator')]
    g_vars = [var for var in trainable_vars if var.name.startswith('generator')]
    
    with tf.control_dependencies(tf.get_collection(tf.GraphKeys.UPDATE_OPS)):
        d_train_op = tf.train.AdamOptimizer(learning_rate=learning_rate, beta1=beta1).minimize(d_loss, var_list=d_vars)
        g_train_op = tf.train.AdamOptimizer(learning_rate=learning_rate, beta1=beta1).minimize(g_loss, var_list=g_vars)
    
    return d_train_op, g_train_op


"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_model_opt(model_opt, tf)

Tests Passed

Neural Network Training¶

Show Output¶

Use this function to show the current output of the generator during training. It will help you determine how well the GANs is training.

"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
import numpy as np

def show_generator_output(sess, n_images, input_z, out_channel_dim, image_mode):
    """
    Show example output for the generator
    :param sess: TensorFlow session
    :param n_images: Number of Images to display
    :param input_z: Input Z Tensor
    :param out_channel_dim: The number of channels in the output image
    :param image_mode: The mode to use for images ("RGB" or "L")
    """
    cmap = None if image_mode == 'RGB' else 'gray'
    z_dim = input_z.get_shape().as_list()[-1]
    example_z = np.random.uniform(-1, 1, size=[n_images, z_dim])

    samples = sess.run(
        generator(input_z, out_channel_dim, False),
        feed_dict={input_z: example_z})

    images_grid = helper.images_square_grid(samples, image_mode)
    pyplot.imshow(images_grid, cmap=cmap)
    pyplot.show()

Train¶

Implement train to build and train the GANs. Use the following functions you implemented:

model_inputs(image_width, image_height, image_channels, z_dim)
model_loss(input_real, input_z, out_channel_dim)
model_opt(d_loss, g_loss, learning_rate, beta1)

Use the show_generator_output to show generator output while you train. Running show_generator_output for every batch will drastically increase training time and increase the size of the notebook. It's recommended to print the generator output every 100 batches.

def train(epoch_count, batch_size, z_dim, learning_rate, beta1, get_batches, data_shape, data_image_mode):
    """
    Train the GAN
    :param epoch_count: Number of epochs
    :param batch_size: Batch Size
    :param z_dim: Z dimension
    :param learning_rate: Learning Rate
    :param beta1: The exponential decay rate for the 1st moment in the optimizer
    :param get_batches: Function to get batches
    :param data_shape: Shape of the data
    :param data_image_mode: The image mode to use for images ("RGB" or "L")
    """
    # TODO: Build Model
    image_width = data_shape[1]
    image_height = data_shape[2]
    out_channel_dim = data_shape[3]
    input_real, input_z, learning_rate_p = model_inputs(image_width=image_width, image_height=image_height, 
                                                      image_channels=out_channel_dim, z_dim=z_dim)
    d_loss, g_loss = model_loss(input_real=input_real, input_z=input_z, out_channel_dim=out_channel_dim)
    d_train_opt, g_train_opt = model_opt(d_loss=d_loss, g_loss=g_loss, learning_rate=learning_rate, beta1=beta1) 
    
    step = 0
    
    with tf.Session() as sess:
        sess.run(tf.global_variables_initializer())
        for epoch_i in range(epoch_count):
            for batch_images in get_batches(batch_size):
                # TODO: Train Model
                batch_images *= 2
                step += 1
                
                batch_z = np.random.uniform(-1, 1, size=[batch_size, z_dim])
                
                _ = sess.run(d_train_opt, 
                             feed_dict={input_real: batch_images, input_z: batch_z, learning_rate_p: learning_rate})
                _ = sess.run(g_train_opt, feed_dict={input_real: batch_images, input_z: batch_z, 
                                                     learning_rate_p: learning_rate})
                
                train_loss_d = sess.run(d_loss, feed_dict={input_real: batch_images, input_z: batch_z})
                train_loss_g = sess.run(g_loss, feed_dict={input_z: batch_z})
            
                print("Epoch {}/{}".format(epoch_i + 1, epoch_count),
                      "Discriminator loss: {:.4f}".format(train_loss_d),
                      "Generator loss: {:.4f}".format(train_loss_g))
                if step == 1 or step % 20 == 0:
                    show_generator_output(sess=sess, n_images=25, input_z=input_z, 
                                      out_channel_dim=out_channel_dim, image_mode=data_image_mode)

MNIST¶

Test your GANs architecture on MNIST. After 2 epochs, the GANs should be able to generate images that look like handwritten digits. Make sure the loss of the generator is lower than the loss of the discriminator or close to 0.

batch_size = 64
z_dim = 100
learning_rate = 0.001
beta1 = 0.3


"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
epochs = 2

mnist_dataset = helper.Dataset('mnist', glob(os.path.join(data_dir, 'mnist/*.jpg')))
with tf.Graph().as_default():
    train(epochs, batch_size, z_dim, learning_rate, beta1, mnist_dataset.get_batches,
          mnist_dataset.shape, mnist_dataset.image_mode)

Epoch 1/2 Discriminator loss: 7.7305 Generator loss: 0.0016

Epoch 1/2 Discriminator loss: 1.8435 Generator loss: 0.5067
Epoch 1/2 Discriminator loss: 1.6031 Generator loss: 0.7777
Epoch 1/2 Discriminator loss: 0.6401 Generator loss: 2.9939
Epoch 1/2 Discriminator loss: 1.1043 Generator loss: 0.9055
Epoch 1/2 Discriminator loss: 0.4053 Generator loss: 10.9843
Epoch 1/2 Discriminator loss: 0.4020 Generator loss: 10.4141
Epoch 1/2 Discriminator loss: 0.4032 Generator loss: 4.1457
Epoch 1/2 Discriminator loss: 1.7728 Generator loss: 0.3135
Epoch 1/2 Discriminator loss: 1.3419 Generator loss: 16.3167
Epoch 1/2 Discriminator loss: 0.6818 Generator loss: 13.0355
Epoch 1/2 Discriminator loss: 0.6987 Generator loss: 7.1742
Epoch 1/2 Discriminator loss: 1.1258 Generator loss: 1.1592
Epoch 1/2 Discriminator loss: 0.4045 Generator loss: 9.9121
Epoch 1/2 Discriminator loss: 0.4441 Generator loss: 4.7448
Epoch 1/2 Discriminator loss: 1.4982 Generator loss: 0.7307
Epoch 1/2 Discriminator loss: 2.1163 Generator loss: 13.9644
Epoch 1/2 Discriminator loss: 0.9109 Generator loss: 3.0821
Epoch 1/2 Discriminator loss: 1.2717 Generator loss: 1.1449
Epoch 1/2 Discriminator loss: 0.7648 Generator loss: 5.7517

Epoch 1/2 Discriminator loss: 1.0785 Generator loss: 1.3863
Epoch 1/2 Discriminator loss: 0.8427 Generator loss: 4.5274
Epoch 1/2 Discriminator loss: 3.4128 Generator loss: 0.2588
Epoch 1/2 Discriminator loss: 2.4902 Generator loss: 8.5376
Epoch 1/2 Discriminator loss: 1.7276 Generator loss: 0.7797
Epoch 1/2 Discriminator loss: 1.1449 Generator loss: 1.5884
Epoch 1/2 Discriminator loss: 1.7056 Generator loss: 0.8056
Epoch 1/2 Discriminator loss: 1.2245 Generator loss: 1.8207
Epoch 1/2 Discriminator loss: 1.8683 Generator loss: 0.4493
Epoch 1/2 Discriminator loss: 2.2825 Generator loss: 6.0373
Epoch 1/2 Discriminator loss: 1.2866 Generator loss: 0.8336
Epoch 1/2 Discriminator loss: 0.9209 Generator loss: 1.0779
Epoch 1/2 Discriminator loss: 0.9772 Generator loss: 2.5208
Epoch 1/2 Discriminator loss: 1.7637 Generator loss: 0.3985
Epoch 1/2 Discriminator loss: 1.5484 Generator loss: 2.4701
Epoch 1/2 Discriminator loss: 2.5522 Generator loss: 0.2091
Epoch 1/2 Discriminator loss: 1.9543 Generator loss: 3.5509
Epoch 1/2 Discriminator loss: 2.1957 Generator loss: 0.2295
Epoch 1/2 Discriminator loss: 1.4994 Generator loss: 2.0616
Epoch 1/2 Discriminator loss: 1.7834 Generator loss: 0.3669

Epoch 1/2 Discriminator loss: 1.3609 Generator loss: 1.1521
Epoch 1/2 Discriminator loss: 1.5061 Generator loss: 0.4912
Epoch 1/2 Discriminator loss: 1.2561 Generator loss: 1.2977
Epoch 1/2 Discriminator loss: 1.9668 Generator loss: 0.3104
Epoch 1/2 Discriminator loss: 1.8014 Generator loss: 1.5901
Epoch 1/2 Discriminator loss: 2.5051 Generator loss: 0.1858
Epoch 1/2 Discriminator loss: 1.6948 Generator loss: 1.9202
Epoch 1/2 Discriminator loss: 1.9709 Generator loss: 0.3205
Epoch 1/2 Discriminator loss: 1.5138 Generator loss: 1.1533
Epoch 1/2 Discriminator loss: 1.7730 Generator loss: 0.3802
Epoch 1/2 Discriminator loss: 1.4485 Generator loss: 1.0733
Epoch 1/2 Discriminator loss: 1.5707 Generator loss: 0.5076
Epoch 1/2 Discriminator loss: 1.6413 Generator loss: 0.9881
Epoch 1/2 Discriminator loss: 1.7453 Generator loss: 0.4172
Epoch 1/2 Discriminator loss: 1.5888 Generator loss: 0.7541
Epoch 1/2 Discriminator loss: 1.5414 Generator loss: 0.5933
Epoch 1/2 Discriminator loss: 1.4981 Generator loss: 0.6218
Epoch 1/2 Discriminator loss: 1.5073 Generator loss: 0.7941
Epoch 1/2 Discriminator loss: 1.5132 Generator loss: 0.5308
Epoch 1/2 Discriminator loss: 1.4996 Generator loss: 0.9772

Epoch 1/2 Discriminator loss: 1.7372 Generator loss: 0.3547
Epoch 1/2 Discriminator loss: 1.6603 Generator loss: 1.2357
Epoch 1/2 Discriminator loss: 1.8949 Generator loss: 0.2863
Epoch 1/2 Discriminator loss: 1.9425 Generator loss: 1.9646
Epoch 1/2 Discriminator loss: 1.9079 Generator loss: 0.2828
Epoch 1/2 Discriminator loss: 1.5325 Generator loss: 1.2793
Epoch 1/2 Discriminator loss: 1.5136 Generator loss: 0.5020
Epoch 1/2 Discriminator loss: 1.4201 Generator loss: 0.9851
Epoch 1/2 Discriminator loss: 1.5217 Generator loss: 0.5315
Epoch 1/2 Discriminator loss: 1.3610 Generator loss: 0.9622
Epoch 1/2 Discriminator loss: 1.5138 Generator loss: 0.5314
Epoch 1/2 Discriminator loss: 1.4970 Generator loss: 0.7995
Epoch 1/2 Discriminator loss: 1.5308 Generator loss: 0.5403
Epoch 1/2 Discriminator loss: 1.4048 Generator loss: 0.8489
Epoch 1/2 Discriminator loss: 1.4991 Generator loss: 0.5667
Epoch 1/2 Discriminator loss: 1.4750 Generator loss: 0.8855
Epoch 1/2 Discriminator loss: 1.5743 Generator loss: 0.4684
Epoch 1/2 Discriminator loss: 1.5176 Generator loss: 1.2415
Epoch 1/2 Discriminator loss: 1.7399 Generator loss: 0.3687
Epoch 1/2 Discriminator loss: 1.5793 Generator loss: 1.2078

Epoch 1/2 Discriminator loss: 1.6023 Generator loss: 0.4430
Epoch 1/2 Discriminator loss: 1.4892 Generator loss: 1.0415
Epoch 1/2 Discriminator loss: 1.6636 Generator loss: 0.3954
Epoch 1/2 Discriminator loss: 1.5977 Generator loss: 1.0768
Epoch 1/2 Discriminator loss: 1.5098 Generator loss: 0.5363
Epoch 1/2 Discriminator loss: 1.3830 Generator loss: 0.8172
Epoch 1/2 Discriminator loss: 1.4988 Generator loss: 0.5603
Epoch 1/2 Discriminator loss: 1.4320 Generator loss: 0.8088
Epoch 1/2 Discriminator loss: 1.5480 Generator loss: 0.5119
Epoch 1/2 Discriminator loss: 1.4879 Generator loss: 0.8381
Epoch 1/2 Discriminator loss: 1.5547 Generator loss: 0.4711
Epoch 1/2 Discriminator loss: 1.4267 Generator loss: 0.8524
Epoch 1/2 Discriminator loss: 1.5202 Generator loss: 0.5365
Epoch 1/2 Discriminator loss: 1.4402 Generator loss: 0.9323
Epoch 1/2 Discriminator loss: 1.5362 Generator loss: 0.4822
Epoch 1/2 Discriminator loss: 1.5213 Generator loss: 0.9697
Epoch 1/2 Discriminator loss: 1.5824 Generator loss: 0.4278
Epoch 1/2 Discriminator loss: 1.4690 Generator loss: 1.2312
Epoch 1/2 Discriminator loss: 1.6445 Generator loss: 0.3769
Epoch 1/2 Discriminator loss: 1.6072 Generator loss: 1.2602

Epoch 1/2 Discriminator loss: 1.6438 Generator loss: 0.4011
Epoch 1/2 Discriminator loss: 1.4789 Generator loss: 1.0339
Epoch 1/2 Discriminator loss: 1.5323 Generator loss: 0.4948
Epoch 1/2 Discriminator loss: 1.5427 Generator loss: 0.9864
Epoch 1/2 Discriminator loss: 1.6192 Generator loss: 0.4701
Epoch 1/2 Discriminator loss: 1.5780 Generator loss: 0.8471
Epoch 1/2 Discriminator loss: 1.5070 Generator loss: 0.5081
Epoch 1/2 Discriminator loss: 1.3833 Generator loss: 0.9465
Epoch 1/2 Discriminator loss: 1.4584 Generator loss: 0.6250
Epoch 1/2 Discriminator loss: 1.4062 Generator loss: 0.8494
Epoch 1/2 Discriminator loss: 1.5090 Generator loss: 0.5382
Epoch 1/2 Discriminator loss: 1.4091 Generator loss: 0.9931
Epoch 1/2 Discriminator loss: 1.3838 Generator loss: 0.5968
Epoch 1/2 Discriminator loss: 1.4070 Generator loss: 0.9076
Epoch 1/2 Discriminator loss: 1.3979 Generator loss: 0.6319
Epoch 1/2 Discriminator loss: 1.3721 Generator loss: 0.8208
Epoch 1/2 Discriminator loss: 1.4591 Generator loss: 0.5173
Epoch 1/2 Discriminator loss: 1.3713 Generator loss: 0.9431
Epoch 1/2 Discriminator loss: 1.3990 Generator loss: 0.6000
Epoch 1/2 Discriminator loss: 1.4024 Generator loss: 1.1068

Epoch 1/2 Discriminator loss: 1.4287 Generator loss: 0.5155
Epoch 1/2 Discriminator loss: 1.3802 Generator loss: 1.0777
Epoch 1/2 Discriminator loss: 1.5961 Generator loss: 0.4071
Epoch 1/2 Discriminator loss: 1.4882 Generator loss: 1.1201
Epoch 1/2 Discriminator loss: 1.5019 Generator loss: 0.4888
Epoch 1/2 Discriminator loss: 1.6307 Generator loss: 0.9194
Epoch 1/2 Discriminator loss: 1.5091 Generator loss: 0.5000
Epoch 1/2 Discriminator loss: 1.4242 Generator loss: 1.0253
Epoch 1/2 Discriminator loss: 1.5243 Generator loss: 0.4742
Epoch 1/2 Discriminator loss: 1.3604 Generator loss: 1.1768
Epoch 1/2 Discriminator loss: 1.4706 Generator loss: 0.4991
Epoch 1/2 Discriminator loss: 1.4100 Generator loss: 1.2128
Epoch 1/2 Discriminator loss: 1.6241 Generator loss: 0.3847
Epoch 1/2 Discriminator loss: 1.4407 Generator loss: 1.2307
Epoch 1/2 Discriminator loss: 1.5079 Generator loss: 0.4793
Epoch 1/2 Discriminator loss: 1.4272 Generator loss: 1.0716
Epoch 1/2 Discriminator loss: 1.3896 Generator loss: 0.5758
Epoch 1/2 Discriminator loss: 1.3052 Generator loss: 1.0638
Epoch 1/2 Discriminator loss: 1.5236 Generator loss: 0.4403
Epoch 1/2 Discriminator loss: 1.4742 Generator loss: 1.4498

Epoch 1/2 Discriminator loss: 1.4947 Generator loss: 0.4529
Epoch 1/2 Discriminator loss: 1.3862 Generator loss: 1.0176
Epoch 1/2 Discriminator loss: 1.3966 Generator loss: 0.5593
Epoch 1/2 Discriminator loss: 1.3028 Generator loss: 1.0265
Epoch 1/2 Discriminator loss: 1.3333 Generator loss: 0.6074
Epoch 1/2 Discriminator loss: 1.3689 Generator loss: 0.9854
Epoch 1/2 Discriminator loss: 1.3369 Generator loss: 0.6384
Epoch 1/2 Discriminator loss: 1.2911 Generator loss: 0.8806
Epoch 1/2 Discriminator loss: 1.3554 Generator loss: 0.6981
Epoch 1/2 Discriminator loss: 1.2669 Generator loss: 0.7676
Epoch 1/2 Discriminator loss: 1.3469 Generator loss: 0.8612
Epoch 1/2 Discriminator loss: 1.3472 Generator loss: 0.6103
Epoch 1/2 Discriminator loss: 1.3376 Generator loss: 1.0755
Epoch 1/2 Discriminator loss: 1.4273 Generator loss: 0.5006
Epoch 1/2 Discriminator loss: 1.4284 Generator loss: 1.3963
Epoch 1/2 Discriminator loss: 1.5389 Generator loss: 0.4295
Epoch 1/2 Discriminator loss: 1.4441 Generator loss: 1.2208
Epoch 1/2 Discriminator loss: 1.4190 Generator loss: 0.5239
Epoch 1/2 Discriminator loss: 1.3148 Generator loss: 0.9248
Epoch 1/2 Discriminator loss: 1.3240 Generator loss: 0.7209

Epoch 1/2 Discriminator loss: 1.3339 Generator loss: 0.6421
Epoch 1/2 Discriminator loss: 1.2703 Generator loss: 1.0579
Epoch 1/2 Discriminator loss: 1.3343 Generator loss: 0.6747
Epoch 1/2 Discriminator loss: 1.3175 Generator loss: 0.8641
Epoch 1/2 Discriminator loss: 1.2822 Generator loss: 0.6887
Epoch 1/2 Discriminator loss: 1.2546 Generator loss: 1.0081
Epoch 1/2 Discriminator loss: 1.3796 Generator loss: 0.5837
Epoch 1/2 Discriminator loss: 1.2499 Generator loss: 1.1882
Epoch 1/2 Discriminator loss: 1.4484 Generator loss: 0.4887
Epoch 1/2 Discriminator loss: 1.3766 Generator loss: 1.3925
Epoch 1/2 Discriminator loss: 1.6008 Generator loss: 0.3804
Epoch 1/2 Discriminator loss: 1.3996 Generator loss: 1.5668
Epoch 1/2 Discriminator loss: 1.4806 Generator loss: 0.4685
Epoch 1/2 Discriminator loss: 1.2935 Generator loss: 1.0942
Epoch 1/2 Discriminator loss: 1.3283 Generator loss: 0.5767
Epoch 1/2 Discriminator loss: 1.2908 Generator loss: 1.0954
Epoch 1/2 Discriminator loss: 1.3620 Generator loss: 0.5844
Epoch 1/2 Discriminator loss: 1.2471 Generator loss: 0.9413
Epoch 1/2 Discriminator loss: 1.2867 Generator loss: 0.6850
Epoch 1/2 Discriminator loss: 1.2594 Generator loss: 0.9483

CelebA¶

Run your GANs on CelebA. It will take around 20 minutes on the average GPU to run one epoch. You can run the whole epoch or stop when it starts to generate realistic faces.

batch_size = 64
z_dim = 100
learning_rate = 0.001
beta1 = 0.3


"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
epochs = 1

celeba_dataset = helper.Dataset('celeba', glob(os.path.join(data_dir, 'img_align_celeba/*.jpg')))
with tf.Graph().as_default():
    train(epochs, batch_size, z_dim, learning_rate, beta1, celeba_dataset.get_batches,
          celeba_dataset.shape, celeba_dataset.image_mode)

Epoch 1/1 Discriminator loss: 4.4708 Generator loss: 0.0276

Epoch 1/1 Discriminator loss: 1.0348 Generator loss: 2.6328
Epoch 1/1 Discriminator loss: 2.1676 Generator loss: 0.3597
Epoch 1/1 Discriminator loss: 0.5853 Generator loss: 6.7561
Epoch 1/1 Discriminator loss: 0.7711 Generator loss: 1.7264
Epoch 1/1 Discriminator loss: 0.9684 Generator loss: 1.0310
Epoch 1/1 Discriminator loss: 0.7046 Generator loss: 9.3230
Epoch 1/1 Discriminator loss: 0.6527 Generator loss: 3.2058
Epoch 1/1 Discriminator loss: 2.5485 Generator loss: 0.2075
Epoch 1/1 Discriminator loss: 1.0171 Generator loss: 15.1245
Epoch 1/1 Discriminator loss: 0.4631 Generator loss: 9.7380
Epoch 1/1 Discriminator loss: 3.8425 Generator loss: 0.0583
Epoch 1/1 Discriminator loss: 2.4438 Generator loss: 13.1127
Epoch 1/1 Discriminator loss: 0.8032 Generator loss: 2.7774
Epoch 1/1 Discriminator loss: 8.3286 Generator loss: 0.0055
Epoch 1/1 Discriminator loss: 1.1836 Generator loss: 7.2938
Epoch 1/1 Discriminator loss: 2.7037 Generator loss: 0.2105
Epoch 1/1 Discriminator loss: 1.7823 Generator loss: 6.3465
Epoch 1/1 Discriminator loss: 2.7507 Generator loss: 0.1928
Epoch 1/1 Discriminator loss: 1.8601 Generator loss: 4.1996

Epoch 1/1 Discriminator loss: 2.9869 Generator loss: 0.1231
Epoch 1/1 Discriminator loss: 1.6432 Generator loss: 2.1493
Epoch 1/1 Discriminator loss: 2.0787 Generator loss: 0.3453
Epoch 1/1 Discriminator loss: 1.4481 Generator loss: 1.1665
Epoch 1/1 Discriminator loss: 2.2175 Generator loss: 0.2558
Epoch 1/1 Discriminator loss: 1.7431 Generator loss: 1.3436
Epoch 1/1 Discriminator loss: 2.4402 Generator loss: 0.1861
Epoch 1/1 Discriminator loss: 1.9195 Generator loss: 1.5918
Epoch 1/1 Discriminator loss: 2.3203 Generator loss: 0.2047
Epoch 1/1 Discriminator loss: 1.8173 Generator loss: 1.7971
Epoch 1/1 Discriminator loss: 2.0593 Generator loss: 0.2524
Epoch 1/1 Discriminator loss: 1.5799 Generator loss: 1.6439
Epoch 1/1 Discriminator loss: 1.9361 Generator loss: 0.3077
Epoch 1/1 Discriminator loss: 1.8154 Generator loss: 1.7861
Epoch 1/1 Discriminator loss: 1.8838 Generator loss: 0.3061
Epoch 1/1 Discriminator loss: 1.5938 Generator loss: 1.0183
Epoch 1/1 Discriminator loss: 1.9422 Generator loss: 0.3205
Epoch 1/1 Discriminator loss: 1.7494 Generator loss: 0.6915
Epoch 1/1 Discriminator loss: 2.0020 Generator loss: 0.4130
Epoch 1/1 Discriminator loss: 1.7212 Generator loss: 0.5950

Epoch 1/1 Discriminator loss: 1.7249 Generator loss: 0.5201
Epoch 1/1 Discriminator loss: 1.5896 Generator loss: 0.7641
Epoch 1/1 Discriminator loss: 1.7602 Generator loss: 0.4060
Epoch 1/1 Discriminator loss: 1.6535 Generator loss: 1.0471
Epoch 1/1 Discriminator loss: 2.1832 Generator loss: 0.2044
Epoch 1/1 Discriminator loss: 2.1953 Generator loss: 2.2408
Epoch 1/1 Discriminator loss: 1.7542 Generator loss: 0.3656
Epoch 1/1 Discriminator loss: 1.5087 Generator loss: 0.7256
Epoch 1/1 Discriminator loss: 1.5772 Generator loss: 0.6730
Epoch 1/1 Discriminator loss: 1.4672 Generator loss: 0.6387
Epoch 1/1 Discriminator loss: 1.6468 Generator loss: 0.6181
Epoch 1/1 Discriminator loss: 1.5815 Generator loss: 0.6298
Epoch 1/1 Discriminator loss: 1.5889 Generator loss: 0.5831
Epoch 1/1 Discriminator loss: 1.6714 Generator loss: 0.5222
Epoch 1/1 Discriminator loss: 1.6310 Generator loss: 0.7514
Epoch 1/1 Discriminator loss: 1.7945 Generator loss: 0.3850
Epoch 1/1 Discriminator loss: 1.8023 Generator loss: 1.1607
Epoch 1/1 Discriminator loss: 2.2589 Generator loss: 0.1774
Epoch 1/1 Discriminator loss: 1.8534 Generator loss: 1.4739
Epoch 1/1 Discriminator loss: 1.7567 Generator loss: 0.3647

Epoch 1/1 Discriminator loss: 1.5855 Generator loss: 0.8555
Epoch 1/1 Discriminator loss: 1.6774 Generator loss: 0.4894
Epoch 1/1 Discriminator loss: 1.5565 Generator loss: 0.7281
Epoch 1/1 Discriminator loss: 1.6646 Generator loss: 0.4901
Epoch 1/1 Discriminator loss: 1.6818 Generator loss: 0.6238
Epoch 1/1 Discriminator loss: 1.6253 Generator loss: 0.5328
Epoch 1/1 Discriminator loss: 1.5664 Generator loss: 0.8547
Epoch 1/1 Discriminator loss: 1.8140 Generator loss: 0.3192
Epoch 1/1 Discriminator loss: 1.7625 Generator loss: 1.4996
Epoch 1/1 Discriminator loss: 1.8721 Generator loss: 0.2843
Epoch 1/1 Discriminator loss: 1.4902 Generator loss: 1.2200
Epoch 1/1 Discriminator loss: 1.7113 Generator loss: 0.4122
Epoch 1/1 Discriminator loss: 1.5644 Generator loss: 0.7868
Epoch 1/1 Discriminator loss: 1.3366 Generator loss: 0.7158
Epoch 1/1 Discriminator loss: 1.2654 Generator loss: 0.9073
Epoch 1/1 Discriminator loss: 1.7115 Generator loss: 0.4181
Epoch 1/1 Discriminator loss: 1.7161 Generator loss: 1.2380
Epoch 1/1 Discriminator loss: 1.8908 Generator loss: 0.2827
Epoch 1/1 Discriminator loss: 1.5629 Generator loss: 1.5697
Epoch 1/1 Discriminator loss: 1.7033 Generator loss: 0.3670

Epoch 1/1 Discriminator loss: 1.4622 Generator loss: 1.3834
Epoch 1/1 Discriminator loss: 1.6230 Generator loss: 0.4390
Epoch 1/1 Discriminator loss: 1.4402 Generator loss: 0.7960
Epoch 1/1 Discriminator loss: 1.5095 Generator loss: 0.5887
Epoch 1/1 Discriminator loss: 1.5253 Generator loss: 0.5492
Epoch 1/1 Discriminator loss: 1.6164 Generator loss: 0.7035
Epoch 1/1 Discriminator loss: 1.5429 Generator loss: 0.4781
Epoch 1/1 Discriminator loss: 1.3494 Generator loss: 1.8428
Epoch 1/1 Discriminator loss: 2.2941 Generator loss: 0.1731
Epoch 1/1 Discriminator loss: 1.5865 Generator loss: 1.7275
Epoch 1/1 Discriminator loss: 1.5704 Generator loss: 0.4465
Epoch 1/1 Discriminator loss: 1.3614 Generator loss: 0.7258
Epoch 1/1 Discriminator loss: 1.6492 Generator loss: 0.5836
Epoch 1/1 Discriminator loss: 1.5573 Generator loss: 0.7337
Epoch 1/1 Discriminator loss: 1.5673 Generator loss: 0.5149
Epoch 1/1 Discriminator loss: 1.4550 Generator loss: 1.1288
Epoch 1/1 Discriminator loss: 1.8202 Generator loss: 0.3040
Epoch 1/1 Discriminator loss: 2.0577 Generator loss: 2.0152
Epoch 1/1 Discriminator loss: 1.7296 Generator loss: 0.3461
Epoch 1/1 Discriminator loss: 1.3602 Generator loss: 1.0281

Epoch 1/1 Discriminator loss: 1.3263 Generator loss: 0.8477
Epoch 1/1 Discriminator loss: 1.2982 Generator loss: 0.7603
Epoch 1/1 Discriminator loss: 1.3826 Generator loss: 0.6792
Epoch 1/1 Discriminator loss: 1.5196 Generator loss: 0.7399
Epoch 1/1 Discriminator loss: 1.3399 Generator loss: 0.7150
Epoch 1/1 Discriminator loss: 1.4539 Generator loss: 0.6090
Epoch 1/1 Discriminator loss: 1.3604 Generator loss: 0.8617
Epoch 1/1 Discriminator loss: 1.2590 Generator loss: 0.7575
Epoch 1/1 Discriminator loss: 1.1678 Generator loss: 0.9366
Epoch 1/1 Discriminator loss: 1.4886 Generator loss: 0.5038
Epoch 1/1 Discriminator loss: 2.0167 Generator loss: 1.8103
Epoch 1/1 Discriminator loss: 2.5933 Generator loss: 0.1366
Epoch 1/1 Discriminator loss: 1.5745 Generator loss: 1.4674
Epoch 1/1 Discriminator loss: 1.5021 Generator loss: 0.5636
Epoch 1/1 Discriminator loss: 1.4983 Generator loss: 0.6129
Epoch 1/1 Discriminator loss: 1.4180 Generator loss: 0.8372
Epoch 1/1 Discriminator loss: 1.5516 Generator loss: 0.5769
Epoch 1/1 Discriminator loss: 1.5053 Generator loss: 0.7547
Epoch 1/1 Discriminator loss: 1.6712 Generator loss: 0.4178
Epoch 1/1 Discriminator loss: 1.7973 Generator loss: 1.5984

Epoch 1/1 Discriminator loss: 2.2722 Generator loss: 0.1686
Epoch 1/1 Discriminator loss: 1.5858 Generator loss: 1.6032
Epoch 1/1 Discriminator loss: 1.5221 Generator loss: 0.4994
Epoch 1/1 Discriminator loss: 1.3148 Generator loss: 1.0636
Epoch 1/1 Discriminator loss: 1.3782 Generator loss: 0.6410
Epoch 1/1 Discriminator loss: 1.4070 Generator loss: 0.7263
Epoch 1/1 Discriminator loss: 1.3407 Generator loss: 0.8014
Epoch 1/1 Discriminator loss: 1.2203 Generator loss: 0.8876
Epoch 1/1 Discriminator loss: 1.5414 Generator loss: 0.4465
Epoch 1/1 Discriminator loss: 1.6700 Generator loss: 1.2479
Epoch 1/1 Discriminator loss: 1.9748 Generator loss: 0.2403
Epoch 1/1 Discriminator loss: 1.7731 Generator loss: 1.3085
Epoch 1/1 Discriminator loss: 1.5524 Generator loss: 0.4457
Epoch 1/1 Discriminator loss: 1.3290 Generator loss: 0.8350
Epoch 1/1 Discriminator loss: 1.4011 Generator loss: 0.6949
Epoch 1/1 Discriminator loss: 1.4218 Generator loss: 0.7390
Epoch 1/1 Discriminator loss: 1.2963 Generator loss: 0.7656
Epoch 1/1 Discriminator loss: 1.2925 Generator loss: 0.8494
Epoch 1/1 Discriminator loss: 1.4355 Generator loss: 0.5250
Epoch 1/1 Discriminator loss: 1.4006 Generator loss: 1.1029

Epoch 1/1 Discriminator loss: 1.6903 Generator loss: 0.3597
Epoch 1/1 Discriminator loss: 1.4572 Generator loss: 1.1502
Epoch 1/1 Discriminator loss: 1.5231 Generator loss: 0.5070
Epoch 1/1 Discriminator loss: 1.4135 Generator loss: 0.6997
Epoch 1/1 Discriminator loss: 1.3316 Generator loss: 0.7413
Epoch 1/1 Discriminator loss: 1.3678 Generator loss: 0.9317
Epoch 1/1 Discriminator loss: 1.6987 Generator loss: 0.3713
Epoch 1/1 Discriminator loss: 1.6366 Generator loss: 1.1642
Epoch 1/1 Discriminator loss: 1.6841 Generator loss: 0.3598
Epoch 1/1 Discriminator loss: 1.6386 Generator loss: 1.0377
Epoch 1/1 Discriminator loss: 1.5981 Generator loss: 0.4429
Epoch 1/1 Discriminator loss: 1.2478 Generator loss: 1.1096
Epoch 1/1 Discriminator loss: 1.5533 Generator loss: 0.4392
Epoch 1/1 Discriminator loss: 1.3536 Generator loss: 0.7593
Epoch 1/1 Discriminator loss: 1.4214 Generator loss: 0.8756
Epoch 1/1 Discriminator loss: 1.5185 Generator loss: 0.5297
Epoch 1/1 Discriminator loss: 1.3260 Generator loss: 0.9670
Epoch 1/1 Discriminator loss: 1.4464 Generator loss: 0.5385
Epoch 1/1 Discriminator loss: 1.4393 Generator loss: 0.9881
Epoch 1/1 Discriminator loss: 1.7624 Generator loss: 0.3438

Epoch 1/1 Discriminator loss: 1.4395 Generator loss: 1.0659
Epoch 1/1 Discriminator loss: 1.6880 Generator loss: 0.3688
Epoch 1/1 Discriminator loss: 1.5487 Generator loss: 1.1511
Epoch 1/1 Discriminator loss: 1.6265 Generator loss: 0.4266
Epoch 1/1 Discriminator loss: 1.4391 Generator loss: 1.1798
Epoch 1/1 Discriminator loss: 1.5962 Generator loss: 0.4645
Epoch 1/1 Discriminator loss: 1.4208 Generator loss: 0.9100
Epoch 1/1 Discriminator loss: 1.5825 Generator loss: 0.5268
Epoch 1/1 Discriminator loss: 1.3613 Generator loss: 0.8913
Epoch 1/1 Discriminator loss: 1.5242 Generator loss: 0.6314
Epoch 1/1 Discriminator loss: 1.3602 Generator loss: 0.7902
Epoch 1/1 Discriminator loss: 1.3924 Generator loss: 0.7018
Epoch 1/1 Discriminator loss: 1.4910 Generator loss: 0.6450
Epoch 1/1 Discriminator loss: 1.4755 Generator loss: 0.5386
Epoch 1/1 Discriminator loss: 1.4822 Generator loss: 0.8425
Epoch 1/1 Discriminator loss: 1.4046 Generator loss: 0.6865
Epoch 1/1 Discriminator loss: 1.6028 Generator loss: 0.5673
Epoch 1/1 Discriminator loss: 1.4430 Generator loss: 0.6525
Epoch 1/1 Discriminator loss: 1.3866 Generator loss: 0.7062
Epoch 1/1 Discriminator loss: 1.4262 Generator loss: 0.6549

Submitting This Project¶

When submitting this project, make sure to run all the cells before saving the notebook. Save the notebook file as "dlnd_face_generation.ipynb" and save it as a HTML file under "File" -> "Download as". Include the "helper.py" and "problem_unittests.py" files in your submission.